CLDR-16855 Survey Tool account creation date #5145

btangmu · 2025-10-17T18:31:14Z

-New column firstdate in db cldr_users table, add and populate if not present when ST starts up

-Already existing users get approximate dates based on the first versions in which they voted, according to vote tables from version 25 and later; users who never voted get null meaning unknown

-When new user accounts are added, firstdate will get the creation date

-Display as (account created ...) in Account Settings, with indication if approximate or unknown

-Rename Java User.last_connect to User.lastlogin for consistency with lastlogin in db cldr_users table

-New convenience function DBUtils.addColumnIfMissing; slightly refactor STFactory to use it

-Change some items in UserRegistry from public to private

-Minor edits for consistency, such as uppercase for SQL keywords

-Fix recently added bug in cldrAddUser.mjs and cldrAddUser.mjs: set orgName if not admin

-Comments

CLDR-16855

This PR completes the ticket.

ALLOW_MANY_COMMITS=true

-New column firstdate in db cldr_users table, add and populate if not present when ST starts up -Already existing users get approximate dates based on the first versions in which they voted, according to vote tables from version 25 and later; users who never voted get null meaning unknown -When new user accounts are added, firstdate will get the creation date -Display as (account created ...) in Account Settings, with indication if approximate or unknown -Rename Java User.last_connect to User.lastlogin for consistency with lastlogin in db cldr_users table -New convenience function DBUtils.addColumnIfMissing; slightly refactor STFactory to use it -Change some items in UserRegistry from public to private -Minor edits for consistency, such as uppercase for SQL keywords -Fix recently added bug in cldrAddUser.mjs and cldrAddUser.mjs: set orgName if not admin -Comments

-Fix comments and a variable name

srl295

Looksgood. Would be good to test a blank-slate startup (with no DB)

btangmu · 2025-10-22T15:29:50Z

Would be good to test a blank-slate startup (with no DB)

Excellent question. I've just tried that both on main branch and on this branch. I tried it two ways: (1) with empty cldrdb; (2) with non-existent cldrdb. In all of these scenarios, including with main branch, ST fails to start.

What's the expected/desired behavior?

I imagine that with an empty db it should start with a clean slate and the tables should get initialized appropriately. With non-existent db, I imagine there may be security/permissions problems with automatic creation of a new db.

btangmu · 2025-10-22T16:59:52Z

With some debugging turned on, I'm seeing contradictions on start-up, on main branch, with empty DB:

[INFO] [WARNING ] table cldr_users already existed.
[INFO] [err] java.sql.SQLSyntaxErrorException: Table 'cldrdb.cldr_users' doesn't exist

btangmu · 2025-10-22T18:45:12Z

@srl295 I suspect a bug involving tableExists. With these changes on a new branch from main,

diff --git a/tools/cldr-apps/src/main/java/org/unicode/cldr/web/DBUtils.java b/tools/cldr-apps/src/main/java/org/unicode/cldr/web/DBUtils.java
index f44c0efe2d..d9448bba05 100644
--- a/tools/cldr-apps/src/main/java/org/unicode/cldr/web/DBUtils.java
+++ b/tools/cldr-apps/src/main/java/org/unicode/cldr/web/DBUtils.java
@@ -63,7 +63,7 @@ public class DBUtils {
     static final Logger logger = SurveyLog.forClass(DBUtils.class);
 
     private static final String DEFAULT_DATA_SOURCE = "java:comp/DefaultDataSource";
-    private static final boolean DEBUG = false;
+    private static final boolean DEBUG = true;
     private static final String JDBC_SURVEYTOOL = ("jdbc/SurveyTool");
     private static DataSource datasource = null;
     private String connectionUrl = null;
@@ -296,6 +296,17 @@ public class DBUtils {
             final boolean hadTable = rs.next();
 
             if (hadTable) {
+                if (DEBUG && table.equals("cldr_users")) {
+                    ResultSetMetaData rsmd = rs.getMetaData();
+                    int columnCount = rsmd.getColumnCount();
+                    // System.out.println("Column Name of 1st column: "+rsmd.getColumnName(1));
+                    // System.out.println("Column Type Name of 1st column: "+rsmd.getColumnTypeName(1));
+
+                    for (int i = 1; i <= columnCount; i++) {
+                        SurveyLog.warnOnce(logger, "canonName = " + canonName + "; rs.getString[" + i + "] = " +
+                            rsmd.getColumnName(i) + " " + rs.getString(i));
+                    }
+                }
                 // flush remaining rows
                 while (rs.next())
                     ;
diff --git a/tools/cldr-apps/src/main/java/org/unicode/cldr/web/UserRegistry.java b/tools/cldr-apps/src/main/java/org/unicode/cldr/web/UserRegistry.java
index c36f4390c8..39d7d61ee2 100644
--- a/tools/cldr-apps/src/main/java/org/unicode/cldr/web/UserRegistry.java
+++ b/tools/cldr-apps/src/main/java/org/unicode/cldr/web/UserRegistry.java
@@ -48,6 +48,9 @@ import org.unicode.cldr.web.util.JSONString;
  * @see UserRegistry.User
  */
 public class UserRegistry {
+
+    private static final boolean DEBUG = true;
+
     public static final String ADMIN_EMAIL = "admin@";
 
     /**
@@ -742,9 +745,15 @@ public class UserRegistry {
             synchronized (conn) {
                 boolean hadUserTable = DBUtils.hasTable(CLDR_USERS);
                 if (!hadUserTable) {
+                    if (DEBUG) {
+                        SurveyLog.warnOnce(logger, "Table " + CLDR_USERS + " does not exist. Calling createUserTable.");
+                    }
                     createUserTable(conn);
                     conn.commit();
                 } else {
+                    if (DEBUG) {
+                        SurveyLog.warnOnce(logger, "Table " + CLDR_USERS + " already exists in UserRegistry.setupDB.");
+                    }
                     /* update table to DATETIME instead of TIMESTAMP */
                     Statement s = conn.createStatement();
                     sql = "alter table cldr_users change lastlogin lastlogin DATETIME";

I'm seeing:

[INFO] [WARNING ] table set_kinds already existed.
[INFO] [WARNING ] table set_values already existed.
[INFO] [WARNING ] canonName = cldr_users; rs.getString[1] = TABLE_CAT cldrtest
[INFO] [WARNING ] canonName = cldr_users; rs.getString[2] = TABLE_SCHEM null
[INFO] [WARNING ] canonName = cldr_users; rs.getString[3] = TABLE_NAME cldr_users
[INFO] [WARNING ] canonName = cldr_users; rs.getString[4] = TABLE_TYPE TABLE
[INFO] [WARNING ] canonName = cldr_users; rs.getString[5] = REMARKS 
[INFO] [WARNING ] canonName = cldr_users; rs.getString[6] = TYPE_CAT null
[INFO] [WARNING ] canonName = cldr_users; rs.getString[7] = TYPE_SCHEM null
[INFO] [WARNING ] canonName = cldr_users; rs.getString[8] = TYPE_NAME null
[INFO] [WARNING ] canonName = cldr_users; rs.getString[9] = SELF_REFERENCING_COL_NAME null
[INFO] [WARNING ] canonName = cldr_users; rs.getString[10] = REF_GENERATION null
[INFO] [WARNING ] table cldr_users already existed.
[INFO] [WARNING ] Table cldr_users already exists in UserRegistry.setupDB.
[INFO] [err] java.sql.SQLSyntaxErrorException: Table 'cldrdb.cldr_users' doesn't exist
[INFO] [err] 	at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120)
[INFO] [err] 	at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
[INFO] [err] 	at com.mysql.cj.jdbc.StatementImpl.executeInternal(StatementImpl.java:763)
[INFO] [err] 	at com.mysql.cj.jdbc.StatementImpl.execute(StatementImpl.java:648)
[INFO] [err] 	at com.mysql.cj.jdbc.StatementWrapper.execute(StatementWrapper.java:529)
[INFO] [err] 	at com.ibm.ws.rsadapter.jdbc.WSJdbcStatement.execute(WSJdbcStatement.java:455)
[INFO] [err] 	at org.unicode.cldr.web.UserRegistry.setupDB(UserRegistry.java:760)

Notice that TABLE_CAT is "cldrtest" NOT "cldrdb". If I run ST again (without emptying the db), TABLE_CAT is "cldrdb" and the table really does exist. So there seems to be some kind of a delayed effect, and the problem involves this name "cldrtest". Do you have any idea where the name "cldrtest" comes from? I don't find it in the Java code.

btangmu · 2025-10-23T16:38:48Z

I made a new ticket for the preexisting empty-DB bug: https://unicode-org.atlassian.net/browse/CLDR-19085

github-actions bot assigned btangmu Oct 17, 2025

btangmu added 2 commits October 17, 2025 14:44

CLDR-16855 Survey Tool account creation date, minor fixes

d8c7ff9

-Fix comments and a variable name

CLDR-16855 Survey Tool account creation date, further minor improvments

ccfaf70

btangmu requested a review from srl295 October 20, 2025 14:08

btangmu marked this pull request as ready for review October 20, 2025 14:08

srl295 approved these changes Oct 21, 2025

View reviewed changes

btangmu merged commit 36f5f36 into unicode-org:main Oct 23, 2025
15 checks passed

btangmu deleted the t16855_b branch October 23, 2025 16:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

CLDR-16855 Survey Tool account creation date #5145

CLDR-16855 Survey Tool account creation date #5145

Uh oh!

btangmu commented Oct 17, 2025

Uh oh!

srl295 left a comment

Uh oh!

btangmu commented Oct 22, 2025

Uh oh!

btangmu commented Oct 22, 2025

Uh oh!

btangmu commented Oct 22, 2025

Uh oh!

Uh oh!

btangmu commented Oct 23, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

CLDR-16855 Survey Tool account creation date #5145

CLDR-16855 Survey Tool account creation date #5145

Uh oh!

Conversation

btangmu commented Oct 17, 2025

Uh oh!

srl295 left a comment

Choose a reason for hiding this comment

Uh oh!

btangmu commented Oct 22, 2025

Uh oh!

btangmu commented Oct 22, 2025

Uh oh!

btangmu commented Oct 22, 2025

Uh oh!

Uh oh!

btangmu commented Oct 23, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants